{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1lg7IjKrMKyCkrqq4xffviw7DaOac-I3y?usp=sharing)"
      ],
      "metadata": {
        "id": "LtPu3yuiEwRe"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# A Simple implementation of the ReAct Agents with Porktey\n",
        "This notebook has been forked from Simon Willison's implementation of simple Python implementation of the ReAct pattern for LLMs-  https://til.simonwillison.net/llms/python-react-pattern.\n",
        "\n",
        "ReACT (Reasoning and Action) agent is a framework designed to integrate the reasoning capabilities of large language models (LLMs) with the ability to take actionable steps, creating a more systematic system that can understand and process information, evaluate decisions, take appropriate actions, communicate responses, and track actions.\n",
        "\n",
        "\n",
        "[Portkey](https://portkey.ai) helps you grain granular insights into your Agents workflow\n",
        "Key Features. By routing your agent's requests through Portkey, you gain access to the following production-grade features:\n",
        "\n",
        "1. **Interoperability**\n",
        "- Easily switch between 200+ LLMs\n",
        "- Call various LLMs (Anthropic, Gemini, Mistral, Azure OpenAI, Google Vertex AI, AWS Bedrock, and more)\n",
        "- Choose the best LLM for your ReAct based agent\n",
        "\n",
        "2. **Observability**\n",
        "- Get granular insights into cost, tokens used, and latency using Portkey for understanding ReAct agents' performance\n",
        "- Helps you identify bottlenecks and optimize resource usage\n",
        "\n",
        "\n",
        "3. **Reliability**\n",
        "-  Long ReAct pipelines with multiple API calls can fail at any stage, disrupting the entire process.\n",
        "- Portkey offers built-in fallbacks between different LLMs or providers\n",
        "- Implements load-balancing across multiple instances or API keys\n",
        "- Provides automatic retries and request timeouts\n",
        "- Makes ReAct more reliable and resilient\n",
        "\n",
        "\n",
        "4. **Caching** (simple & semantic)\n",
        "- Store frequently accessed data to reduce expensive API calls\n",
        "- Portkey offers Simple and Semantics caching for your ReAct agents.\n",
        "\n",
        "5. **Logs**\n",
        "- ReAct run is complex. Logs are essential for diagnosing issues, understanding LLMs behavior, and improving performance.\n",
        "- Portkey provides detailed records of actions and tool use\n",
        "- Crucial for debugging and managing complex ReAct runs\n",
        "\n",
        "6. **Prompt Management**\n",
        "- Manage ReAct Prompts in Portkey's Prompt Library\n",
        "- Try new versions of Prompts without changing code\n",
        "- Utilize Portkey's inbuilt Prompt Management Suite\n",
        "\n",
        "\n",
        "For more information on how to use these features please refer to the [Portkey documentation](https://docs.portkey.ai/docs)."
      ],
      "metadata": {
        "id": "_NMXSDOYB-hi"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Installing Dependencies"
      ],
      "metadata": {
        "id": "-Xu2ANcSDezG"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install portkey-ai openai"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "collapsed": true,
        "id": "CNV8tqDc9Ggc",
        "outputId": "3e1354b7-970b-43ba-c921-2b5d6cb99923"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Collecting portkey-ai\n",
            "  Downloading portkey_ai-1.7.1-py3-none-any.whl.metadata (7.3 kB)\n",
            "Collecting openai\n",
            "  Downloading openai-1.37.1-py3-none-any.whl.metadata (22 kB)\n",
            "Collecting httpx (from portkey-ai)\n",
            "  Downloading httpx-0.27.0-py3-none-any.whl.metadata (7.2 kB)\n",
            "Collecting mypy<2.0,>=0.991 (from portkey-ai)\n",
            "  Downloading mypy-1.11.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl.metadata (1.9 kB)\n",
            "Requirement already satisfied: typing-extensions<5.0,>=4.7.1 in /usr/local/lib/python3.10/dist-packages (from portkey-ai) (4.12.2)\n",
            "Requirement already satisfied: pydantic>=1.10.12 in /usr/local/lib/python3.10/dist-packages (from portkey-ai) (2.8.2)\n",
            "Requirement already satisfied: anyio<5,>=3.5.0 in /usr/local/lib/python3.10/dist-packages (from portkey-ai) (3.7.1)\n",
            "Requirement already satisfied: distro<2,>=1.7.0 in /usr/lib/python3/dist-packages (from portkey-ai) (1.7.0)\n",
            "Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from portkey-ai) (1.3.1)\n",
            "Collecting cached-property (from portkey-ai)\n",
            "  Downloading cached_property-1.5.2-py2.py3-none-any.whl.metadata (11 kB)\n",
            "Requirement already satisfied: tqdm>4 in /usr/local/lib/python3.10/dist-packages (from portkey-ai) (4.66.4)\n",
            "Collecting types-requests (from portkey-ai)\n",
            "  Downloading types_requests-2.32.0.20240712-py3-none-any.whl.metadata (1.9 kB)\n",
            "Requirement already satisfied: idna>=2.8 in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.5.0->portkey-ai) (3.7)\n",
            "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.5.0->portkey-ai) (1.2.2)\n",
            "Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from httpx->portkey-ai) (2024.7.4)\n",
            "Collecting httpcore==1.* (from httpx->portkey-ai)\n",
            "  Downloading httpcore-1.0.5-py3-none-any.whl.metadata (20 kB)\n",
            "Collecting h11<0.15,>=0.13 (from httpcore==1.*->httpx->portkey-ai)\n",
            "  Downloading h11-0.14.0-py3-none-any.whl.metadata (8.2 kB)\n",
            "Collecting mypy-extensions>=1.0.0 (from mypy<2.0,>=0.991->portkey-ai)\n",
            "  Downloading mypy_extensions-1.0.0-py3-none-any.whl.metadata (1.1 kB)\n",
            "Requirement already satisfied: tomli>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from mypy<2.0,>=0.991->portkey-ai) (2.0.1)\n",
            "Requirement already satisfied: annotated-types>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from pydantic>=1.10.12->portkey-ai) (0.7.0)\n",
            "Requirement already satisfied: pydantic-core==2.20.1 in /usr/local/lib/python3.10/dist-packages (from pydantic>=1.10.12->portkey-ai) (2.20.1)\n",
            "Requirement already satisfied: urllib3>=2 in /usr/local/lib/python3.10/dist-packages (from types-requests->portkey-ai) (2.0.7)\n",
            "Downloading portkey_ai-1.7.1-py3-none-any.whl (405 kB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m406.0/406.0 kB\u001b[0m \u001b[31m8.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading openai-1.37.1-py3-none-any.whl (337 kB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m337.0/337.0 kB\u001b[0m \u001b[31m20.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading httpx-0.27.0-py3-none-any.whl (75 kB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m75.6/75.6 kB\u001b[0m \u001b[31m4.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading httpcore-1.0.5-py3-none-any.whl (77 kB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m77.9/77.9 kB\u001b[0m \u001b[31m4.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading mypy-1.11.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl (12.5 MB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m12.5/12.5 MB\u001b[0m \u001b[31m65.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading cached_property-1.5.2-py2.py3-none-any.whl (7.6 kB)\n",
            "Downloading types_requests-2.32.0.20240712-py3-none-any.whl (15 kB)\n",
            "Downloading mypy_extensions-1.0.0-py3-none-any.whl (4.7 kB)\n",
            "Downloading h11-0.14.0-py3-none-any.whl (58 kB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m2.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hInstalling collected packages: cached-property, types-requests, mypy-extensions, h11, mypy, httpcore, httpx, portkey-ai, openai\n",
            "Successfully installed cached-property-1.5.2 h11-0.14.0 httpcore-1.0.5 httpx-0.27.0 mypy-1.11.1 mypy-extensions-1.0.0 openai-1.37.1 portkey-ai-1.7.1 types-requests-2.32.0.20240712\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Importing Libraries"
      ],
      "metadata": {
        "id": "rDScTDRhDjAM"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import re\n",
        "import httpx\n",
        "\n",
        "from openai import OpenAI\n",
        "from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders\n"
      ],
      "metadata": {
        "id": "tTtPkKnP7vR5"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Creating A Basic ReAct agent with Portkey"
      ],
      "metadata": {
        "id": "VqStK8BaDxO8"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "client = OpenAI(\n",
        "    api_key=\"OPENAI_API_KEY\", #Enter your OpenAI API Key\n",
        "    base_url=PORTKEY_GATEWAY_URL,\n",
        "    default_headers=createHeaders(\n",
        "        provider=\"openai\",\n",
        "        api_key=\"PORTKEY_API_KEY\", #Enter your Portkey API Key\n",
        "        trace_id=\"react-agent\" #Trace ID for agent\n",
        "    )\n",
        ")\n",
        "\n",
        "\n",
        "class ChatBot:\n",
        "    def __init__(self, system=\"\"):\n",
        "        self.system = system\n",
        "        self.messages = []\n",
        "        if self.system:\n",
        "            self.messages.append({\"role\": \"system\", \"content\": system})\n",
        "\n",
        "    def __call__(self, message):\n",
        "        self.messages.append({\"role\": \"user\", \"content\": message})\n",
        "        result = self.execute()\n",
        "        self.messages.append({\"role\": \"assistant\", \"content\": result})\n",
        "        return result\n",
        "\n",
        "    def execute(self):\n",
        "        completion = client.chat.completions.create(model=\"gpt-3.5-turbo\", messages=self.messages)\n",
        "\n",
        "        return completion.choices[0].message.content\n",
        "\n",
        "prompt = \"\"\"\n",
        "You run in a loop of Thought, Action, PAUSE, Observation.\n",
        "At the end of the loop you output an Answer\n",
        "Use Thought to describe your thoughts about the question you have been asked.\n",
        "Use Action to run one of the actions available to you - then return PAUSE.\n",
        "Observation will be the result of running those actions.\n",
        "\n",
        "Your available actions are:\n",
        "\n",
        "calculate:\n",
        "e.g. calculate: 4 * 7 / 3\n",
        "Runs a calculation and returns the number - uses Python so be sure to use floating point syntax if necessary\n",
        "\n",
        "wikipedia:\n",
        "e.g. wikipedia: Django\n",
        "Returns a summary from searching Wikipedia\n",
        "\n",
        "simon_blog_search:\n",
        "e.g. simon_blog_search: Django\n",
        "Search Simon's blog for that term\n",
        "\n",
        "Always look things up on Wikipedia if you have the opportunity to do so.\n",
        "\n",
        "Example session:\n",
        "\n",
        "Question: What is the capital of France?\n",
        "Thought: I should look up France on Wikipedia\n",
        "Action: wikipedia: France\n",
        "PAUSE\n",
        "\n",
        "You will be called again with this:\n",
        "\n",
        "Observation: France is a country. The capital is Paris.\n",
        "\n",
        "You then output:\n",
        "\n",
        "Answer: The capital of France is Paris\n",
        "\"\"\".strip()\n",
        "\n",
        "\n",
        "action_re = re.compile('^Action: (\\w+): (.*)$')\n",
        "\n",
        "def query(question, max_turns=5):\n",
        "    i = 0\n",
        "    bot = ChatBot(prompt)\n",
        "    next_prompt = question\n",
        "    while i < max_turns:\n",
        "        i += 1\n",
        "        result = bot(next_prompt)\n",
        "        print(result)\n",
        "        actions = [action_re.match(a) for a in result.split('\\n') if action_re.match(a)]\n",
        "        if actions:\n",
        "            # There is an action to run\n",
        "            action, action_input = actions[0].groups()\n",
        "            if action not in known_actions:\n",
        "                raise Exception(\"Unknown action: {}: {}\".format(action, action_input))\n",
        "            print(\" -- running {} {}\".format(action, action_input))\n",
        "            observation = known_actions[action](action_input)\n",
        "            print(\"Observation:\", observation)\n",
        "            next_prompt = \"Observation: {}\".format(observation)\n",
        "        else:\n",
        "            return\n",
        "\n",
        "\n",
        "def wikipedia(q):\n",
        "    return httpx.get(\"https://en.wikipedia.org/w/api.php\", params={\n",
        "        \"action\": \"query\",\n",
        "        \"list\": \"search\",\n",
        "        \"srsearch\": q,\n",
        "        \"format\": \"json\"\n",
        "    }).json()[\"query\"][\"search\"][0][\"snippet\"]\n",
        "\n",
        "\n",
        "def simon_blog_search(q):\n",
        "    results = httpx.get(\"https://datasette.simonwillison.net/simonwillisonblog.json\", params={\n",
        "        \"sql\": \"\"\"\n",
        "        select\n",
        "          blog_entry.title || ': ' || substr(html_strip_tags(blog_entry.body), 0, 1000) as text,\n",
        "          blog_entry.created\n",
        "        from\n",
        "          blog_entry join blog_entry_fts on blog_entry.rowid = blog_entry_fts.rowid\n",
        "        where\n",
        "          blog_entry_fts match escape_fts(:q)\n",
        "        order by\n",
        "          blog_entry_fts.rank\n",
        "        limit\n",
        "          1\"\"\".strip(),\n",
        "        \"_shape\": \"array\",\n",
        "        \"q\": q,\n",
        "    }).json()\n",
        "    return results[0][\"text\"]\n",
        "\n",
        "def calculate(what):\n",
        "    return eval(what)\n",
        "\n",
        "known_actions = {\n",
        "    \"wikipedia\": wikipedia,\n",
        "    \"calculate\": calculate,\n",
        "    \"simon_blog_search\": simon_blog_search\n",
        "}"
      ],
      "metadata": {
        "id": "6M-Cxvz5DtEv"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Asking the Query and Output"
      ],
      "metadata": {
        "id": "kvw-VdoQEPNI"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "G4W2z1m07ufR",
        "outputId": "97fda9e5-fe12-4bc5-e949-71a023dc2c72"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Thought: I should look up India on Wikipedia to find information about its neighboring countries.\n",
            "Action: wikipedia: India\n",
            "PAUSE\n",
            " -- running wikipedia India\n",
            "Observation: <span class=\"searchmatch\">India</span>, officially the Republic of <span class=\"searchmatch\">India</span> (ISO: Bhārat Gaṇarājya), is a country in South Asia. It is the seventh-largest country by area; the most populous\n",
            "Answer: India shares its borders with several countries in South Asia.\n"
          ]
        }
      ],
      "source": [
        "query(\"What does India share borders with?\")"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Observability with Portkey\n"
      ],
      "metadata": {
        "id": "_3NOpSe1FL35"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "JVQm-UoVlOF3"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "By routing requests through Portkey you can track a number of metrics like - tokens used, latency, cost, etc.\n",
        "\n",
        "Here's a screenshot of the dashboard you get with Portkey!\n",
        "\n",
        "![portkey_view.JPG](https://portkey.ai/blog/content/images/2024/07/Screenshot-2024-07-01-at-12.38.28-PM.png)\n",
        "\n",
        "## **Logs**\n",
        "![Presentation design.png]()"
      ],
      "metadata": {
        "id": "wNnQkUyrE_wd"
      }
    }
  ]
}
